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Abstract 

Background: There is no consensus on which protocols should be used to assess foot and lower limb 
biomechanics in clinical practice. The reliability of many assessments has been questioned by previous research. 
The aim of this investigation was to (i) identify (through consensus) what biomechanical examinations are used in 
clinical practice and (ii) evaluate the inter-assessor reliability of some of these examinations. 

Methods: Parti: Using a modified Delphi technique 12 podiatrists derived consensus on the biomechanical 
examinations used in clinical practice. Part 2: Eleven podiatrists assessed 6 participants using a subset of the 
assessment protocol derived in Part 1. Examinations were compared between assessors. 

Results: Clinicians choose to estimate rather than quantitatively measure foot position and motion. Poor 
inter-assessor reliability was recorded for all examinations. Intra-class correlation coefficient values (ICC) for relaxed 
calcaneal stance position were less than 0.23 and were less than 0.14 for neutral calcaneal stance position. For the 
examination of ankle joint dorsiflexion, ICC values suggest moderate reliability (less than 0.61). The results of a 
random effects ANOVA highlight that participant (up to 5.7°), assessor (up to 5.8°) and random (up to 5.7°) error all 
contribute to the total error (up to 9.5° for relaxed calcaneal stance position, up to 10.7° for the examination of 
ankle joint dorsiflexion). Kappa Fleiss values for categorisation of first ray position and mobility were less than 0.05 
and for limb length assessment less than 0.02, indicating slight agreement. 

Conclusion: Static biomechanical assessment of the foot, leg and lower limb is an important protocol in clinical 
practice, but the key examinations used to make inferences about dynamic foot function and to determine orthotic 
prescription are unreliable. 



Background 

Abnormal foot and ankle biomechanics are implicated 
in a wide range of foot and lower limb disorders mana- 
ged by podiatry and other allied health professions [1]. 
Exercise and foot orthotic treatment approaches are 
commonly based on an assessment of a patient's foot 
biomechanics [2]. Protocols for clinical assessment of 
foot biomechanics are broadly based on those advo- 
cated by Root et al. [3,4] and more recent literature 
[5,6]. Root et al. [3,4] proposed a conceptual frame- 
work describing normal and abnormal foot function 
during walking and an assessment protocol that 
enables a clinician to predict the function of the foot 
during walking via a static (i.e. standing or non weight 
bearing) assessment of the foot (the Root static foot 
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assessment protocol). Understanding the reliability of 
an assessment protocol aims to identify whether exam- 
inations are consistent between assessors and across 
time [7] (when there is no change in the status of the 
foot). Good reliability is the basis for sound profes- 
sional practice and is essential for quantifying the value 
of an examination [7]. There is already evidence that 
some or all static foot assessment protocols are unreli- 
able [8-12]. However, most studies have tested only part 
of the assessment protocol described by Root et al. [3] 
and have largely adopted the examinations as they were 
first described [13]. In reality the current implementa- 
tion of the protocol for static foot assessment is influ- 
enced by many factors, including national or local 
professional knowledge (via discussion at workshops/ 
conferences), clinical experience (clinicians would adapt 
their practice to their learning), and practical con- 
straints (time available for an assessment, the range of 
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orthotic prescriptions available to a clinician, and the 
particular profile of patients the clinician sees in their 
practice). Thus, the reliability of static foot assessment 
protocols as they are currently used in practice has not 
been evaluated. 

Relaxed and 'neutral' calcaneal stance position (RCSP 
and NCSP) are arguably the core elements of the Root 
et al. [3] static foot assessment and directly influence 
orthotic prescription. Their importance to practice is 
reflected in the fact that they have been subject to con- 
siderable scrutiny by the physical therapy and related 
communities [8-14]. Menz [8] highlighted how the as- 
sessment is prone to erroneous subjectivity due to skin 
movement artefact, pen marker thickness and practi- 
tioner dexterity. Menz and Keenan [9] examined the 
inter-assessor reliability of a gravity angle finder to 
measure NCSP and RCSP. Pearson's correlation coeffi- 
cient values (and standard error of measurement (SEM)) 
were 0.367 (±3.77°) and 0.742 (±6.27°) respectively. Use 
of a digital goniometer did not significantly improve 
measurements, with correlation (r) values of 0.558 
(±8.47°) and 0.742 (±6.47°) respectively. Keenan and 
Bach [12] reported 95 % confidence intervals of 5.1° 
(range -9.0° to 7.0°) for RCSP and 4.1° (range -2.0° to 
13.0°) for NCSP over two measurement sessions. Both 
studies conclude that the large variation between asses- 
sors would affect diagnosis and treatment rationale. 

Rome [15] highlighted the difficulty of assessing the 
sagittal plane motion of the ankle joint. The poor align- 
ment of the goniometer, non-identification of bony land- 
marks and the variation in force applied would all 
contribute to error [15]. Elveru et al. [16] recorded an 
ICC value of 0.50 and Jonson and Gross [1] an ICC of 
0.65 when examining the inter-assessor reliability of 
assessing ankle joint dorsiflexion with a goniometer. The 
greater reliability in the latter study may be due to 
Jonson and Gross [1] allowing participants to maximally 
dorsiflex their foot rather than a clinician manipulate 
the foot. 

The measurement of first ray mobility and position 
has been measured directly (e.g. mm) and categorically 
(e.g. classification of the range of motion or the position 
of the first ray). Glascoe et al. [17] reported very poor 
inter-assessor reliability for the direct measurement of 
first ray mobility using a ruler, with an ICC value of 
0.05. Similarly Cornwall et al. [18] observed poor agree- 
ment and inter-assessor reliability for the classification 
of first ray mobility, with only 12.5 % agreement for clas- 
sification of first ray mobility as hypomobile and 25.0 % 
agreement for hypermobile. 

There are two approaches to limb length examination: 
direct measures (e.g. tape measure), [19,20] and in-direct 
methods such as palpation of bony pelvic landmarks and 
placing blocks under the heel of the participant [21]. 



The latter appear to have greater reliability [21]. 
Woerman and Binder-MacLeod [21] recorded small 
mean differences (less than 4.3 mm) across five assessors 
when palpating the iliac crest and placing small blocks 
under the heel of the participant to measure the differ- 
ences in limb length. Jonson and Gross [1] recorded 
good inter-assessor reliability (ICC = 0.70) when placing 
blocks under the heel and using a levelling device to as- 
certain pelvis obliquity. 

Understanding how foot biomechanics are assessed 
in current practice, and the reliability of the assess- 
ments, enables us to understand: (1) whether current 
practices have changed since Root et al. [3] first 
introduced their work; and (2) the credibility of the 
assessment protocols used in current practice. This 
project aimed to: (i) identify (through consensus) what 
biomechanical examinations are used in clinical prac- 
tice and (ii) evaluate the inter-assessor reliability of a 
subset of these assessments. 

Methods 

Part 1 Identification of biomechanics assessment 
protocols used in practice 

Twelve podiatrists (working in state funded and private 
health care settings, six male, mean age 42) specialising 
in foot and ankle biomechanics were invited to partici- 
pate. All worked within a specialist biomechanics/mus- 
culoskeletal clinic and had at least 3 years clinical 
experience at this specialist level. Ethical approval was 
granted (University of Salford Institutional Committee) 
and all participants gave written consent. A Delphi 
method [22] was chosen to derive consensus on a foot 
biomechanics assessment protocol. The Delphi method 
[22] is a systematic and structured examination tech- 
nique involving a panel of experts. The method com- 
bines use of questionnaires and group discussion to 
derive consensus [22]. 

There were three keys phases to the development of a 
consensus. 

Phase 1: Questionnaire. All podiatrists answered a 
questionnaire (Additional file 1) anonymously and with- 
out discussion. The questionnaire (written by HJ and 
PB) investigated the use of static foot, leg and lower limb 
biomechanical examinations and gait analysis protocols 
by each podiatrist. Questions were derived from Root 
et al. [3], current undergraduate syllabus, information 
from Valmassey [23] and Michaud [24]. There was also 
space provided for podiatrists to report any additional 
examinations used. 

Most questions required Yes/No answers and required 
information on how often each examination was used, 
the method and whether the information was used to 
classify foot type and/or to develop a treatment rationale. 
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Phase 2: Development of draft consensus from results 
of the questionnaire. From the completed question- 
naires, PB and HJ identified where there was both agree- 
ment and disagreement amongst the expert panel. 
Agreement existed when there was an identifiable trend 
amongst podiatrists, for example the majority of podia- 
trists used the same measurement technique. Disagree- 
ment was where there was poor consensus between 
podiatrists, for example less than half used a particular 
examination. A separate adjudicator (CN) was present 
throughout. A draft assessment protocol was developed 
based on the questionnaire responses. 

Phase 3: Group discussion. A group discussion (led by 
PB, HJ took notes) explored the validity of the question- 
naire results and draft foot assessment protocol from 
Phase 2. Discussion orientated around whether it was 
true reflection of the current practice of the panel mem- 
bers but also related professional disciplines. The areas 
of agreement and disagreement from the questionnaire 
results were elaborated upon though open discussion. 
Podiatrists explained in more detail their assessment 
methodology, their conceptual understanding of the nor- 
mative basis to which pathological cases are compared 
and the rationale for their assessment plan. 

Part 2 Evaluation of the inter-assessor reliability of the 
biomechanical assessment protocol 

Eleven podiatrists (working in state funded and private 
health care settings, six male, mean age 46) specialising 
in foot and ankle biomechanics practice volunteered to 
participate. All worked within a specialist biomechanics/ 
musculoskeletal clinic and had at least 5 years clinical 
experience at this specialist level. 

Each podiatrist assessed six asymptomatic participants 
(three male, mean age 25, mean body mass index [BMI] 
23), using a subset of the assessment protocol defined in 
Part 1 of the study. Ethical approval was granted from 
the University of Salford Institutional Committee and all 
participants gave written consent. This investigation was 
conducted nine months after Part 1. 

Four of the eight biomechanical examination proce- 
dures identified in Part 1 were selected for the inter- 
assessor reliability study. These were selected primarily 
because the podiatrists identified them as essential ra- 
ther than optional components of their clinical assess- 
ment. However, they also provided some assessment of 
the lower limb as well as the foot and could be com- 
pleted within a reasonable time frame. The four assess- 
ments selected by podiatrists were used for all or the 
majority of patients and provided information critical to 
the development of treatment rationale and orthotic pre- 
scription. Thus the four selected contributed more to 
clinical practice than the four assessments omitted. 



The assessments used in the inter-assessor study were 
1) NCSP and RCSP, 2) ankle joint range of dorsiflexion, 
3) first ray mobility and position, and 4) limb length 
examination. They were assessed quantitatively or quali- 
tatively according to the preferences identified in Part 1. 
To help maintain consistency in how the 11 podiatrists 
implemented the assessment protocol, an information 
sheet and demonstration was provided. The participants 
whose feet were to be assessed were placed in six separ- 
ate cubicles at the University clinic. Assessments were 
conducted as per the protocols described in Table 1 and 
podiatrists were allocated 30 minutes to assess each par- 
ticipant and at least 30 minutes rest between each as- 
sessment. In accordance with clinical practice, each 
assessment was completed once for each foot. Each po- 
diatrist recorded their assessments in a booklet. No dis- 
cussion was allowed between podiatrists or participants 
during the assessments. All pen marker lines on the par- 
ticipants were removed between podiatrists. 

Statistical analysis 

The researchers were blind to the data in each booklet. 
All data was collated into Microsoft Excel and then pro- 
cessed through Statistical Package Social Science Soft- 
ware (Version 17.0) (SPSS, Chicago, Illinois, USA). The 
mean, range, standard deviation (SD) and 95 % confi- 
dence intervals (95 % CI) were calculated for NCSP, 
RCSP and the range of ankle joint dorsiflexion. 

Inter-assessor reliability for RCSP, NCSP and the range 
of ankle joint dorsiflexion were calculated using ICC 
(2,1) in accordance with Rankin and Stokes [25]. ICC 
values were chosen as they assess the consistency of 
quantitative measurements made by multiple testers 
(clinicians) measuring the same objects (participants) 
[7]. Bruton et al. [7] suggest that ICC values should not 
be interpreted clinically in isolation. Therefore a random 
effects ANOVA (analysis of variance, crossed random 
effects model) [26] was used to enable further evaluation 
of reliability. A random effects ANOVA models y as a 
constant, plus a random effect due to the assessor (clin- 
ician), a random effect due to the participant (e.g. moved 
their feet) and an overall random error of the examin- 
ation itself. (E 1 assessor error, E 2 participant error, E ran- 
dom error). 

y = u + VE 1 + E 2 + E 

This calculates the extent of between participant vari- 
ability, between assessor variability and the amount of 
random error in the examination. This provides an indi- 
cation of where the majority of error occurs. Therefore 
for each part of the assessment (e.g. NCSP, RCSP), the 
error variables have to be accounted for in addition to 
the true value of the feature being assessed: 
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Table 1 The examination methods used by podiatrists in current practice (identified from Phase 1,2 and 3) 



Biomechanical examination 



Method 



Neutral calcaneal stance position (NCSP) and relaxed 
calcaneal stance position (RCSP) 



Range of ankle joint dorsiflexion 



Position and mobility at the first ray 



Forefoot to rearfoot relationship (frontal plane) 



Range of motion at the first MTPJ 



Foot Posture Index 
Limb length examination 

Visual gait analysis 



(i) Participant standing (ii) Position both feet into NCSP (iii) Pen marker bisection 
line drawn onto the posterior aspect of the calcaneus on both feet (iv) Measurement 
recorded using digital biometer for right foot (v) Identify if calcaneus is positioned 
varus or valgus (vi) Repeat procedure with left foot (vii)Both feet resume RCSP, 
measurement of the bisection line using a digital biometer for both feet 

(i) Participant supine and sitting with back straight against plinth (ii) A straight 
reference line is drawn onto the lateral aspect of leg indicating where one of 
the tractograph arms should be positioned (iii) Tractograph is positioned with 
one lever arm running parallel to the lateral aspect of leg and the other positioned 
parallel to the plantar aspect of the foot running distally (iii) With the knee joint 
extended, the foot is maximally dorsiflexed and the measurement on the tractograph 
recorded (iv) The knee joint is held in a flexed position, the ankle joint is maximally 
dorsiflexed and the measurement on the tractograph recorded (v) Repeat 
procedure with other foot 

(i) Participant supine and sitting with back straight against plinth 

(ii) First ray position classification (dorsiflexed/plantarflexed or neutral) 

(iii) First ray mobility classification (flexible/rigid/normal) (iv) Repeat procedure 
with cother foot 

(i) Participant prone, lying down (ii) Raise one side of pelvis from couch with a 
cushion/pillow, so that the long axis of the contra lateral foot is vertical 
(iii) Position the foot in subtalar joint neutral (iv) Visually observe position of 
forefoot relative to rearfoot. Categorise if neutral/everted/inverted.fv) Repeat 
procedure with other foot 

(i) Participant supine and sitting with back straight against plinth with legs 
extended in front (ii) Place arms of the goniometer parallel to the long 
axis of the first metatarsal and the proximal phalanx of the hallux 
(iii) Manually dorsiflex first MTPJ with first ray free to move and measure 
range of motion with goniometer (iv) Repeat stage 3, with first ray held 
in a neutral position (v) Repeat procedure with left foot 

The 6 Point FPI is to be used and follows the protocol described by 
Redmond et al. [31] 

(i) Participant standing in RCSP (ii) Both ASIS are palpated, identification 
of whether a limb length discrepancy is present (iii) Classification of 
which leg is longer and whether this is less than 5 mm, more than 
5 mm or more than 10 mm 

On conducting a clinical gait analysis, key determinants to be observed 
(i) Position of foot at heel strike (ii) Forefoot and midfoot position during 
loading phase, (iii) Foot position and motion during propulsion 
and re-supination (iv) Movement of the foot and leg during swing phase 
(v) Motion of the hip and knee (vi) Timing and magnitude of motion 



Value provided by the assessor = actual value + (asses- 
sor error (E 1 ) + participant error (E 2 ) + random error (E)). 

A particular advantage of the random effects ANOVA is 
that the outcomes are expressed in the same units as the 
measurement and thus are easily interpreted in terms of 
clinical practice. In addition, the three sources of error 
can be combined to provide an indication of the total 
error due to participant, assessor and random error: 

Total error = V(assessor error (E 1 ) + participant error 
(E 2 ) + random error (E). 

The assessment of first ray mobility and position and 
limb length involved categorical data, therefore the per- 
centage agreement (%) [27] and a Fleiss Kappa [28,29] 
were chosen. 

Percentage agreement can lack sensitivity as it does not 
adjust for that agreement occurring by chance [27]. A 
Fleiss Kappa calculates the reliability of agreement between 



a fixed number of assessors [28-30] and is a better repre- 
sentation of true inter-assessor reliability [27]. Fleiss Kappa 
values range from <0 for poor agreement to 1.00 for per- 
fect agreement [28,30,31]. Both of these statistical mea- 
sures are consistent with the available literature [17,18]. 

Results 

Part 1 

Tables 1 and 2 represent the results of the questionnaire 
and the group discussion. Three key trends were derived 
from the questionnaire (Phases 1 and 2) and formed the 
basis to the subsequent discussion (Phase 3). These 
were: 

(i) The main basis to biomechanical assessment of the 
foot and ankle is the description provided by Root 
et al. [3,4]. 



Jarvis et at. Journal of Foot and Ankle Research 201 2, 5:1 4 
http://www.jfootankleres.eom/content/5/1/14 



Page 5 of 10 



Table 2 Results of Phase 1, 2 (questionnaires) and 3 (group discussion) on assessment of the foot and ankle 



Biomechanical No. of podiatrists Key features of examination 

examination that use the (derived from questionnaire) 

assessment 

(total =12) 



Consensus from group discussion 



NCSP and RCSP 



Range of motion at the 12 
ankle joint 



Range of motion at the 1 1 
subtalar joint 



Position and mobility of 1 1 
the first ray 



Forefoot to rearfoot 1 1 
relationship 

Range of motion at the 1 1 
first MTPJ 



Foot Posture Index 
(FPI) [31] 



Assessment of the 
lower limb 



12 



Leg length discrepancy 7 to 1 
examination 



Additional biomechanical NA 
examinations 



Gait Analysis 



Key determinants of the NA 
gait cycle to be observed 
during a routine gait 
analysis 



Position is estimated not measured 9/9 Use this Frontal plane position of the calcaneus relative to 



as an assessment of foot type 8/9 Use this to 
develop a treatment rationale (for example 
orthotic prescription) 



Range of motion is estimated, not measured 
12/12 podiatrists assessed with the knee 
extended 

9/12 podiatrists assessed with the knee flexed 
The total range of motion and range of 
dorsiflexion are assessed 



Motion is estimated not measured Subtalar 
joint neutral (non weight-bearing) is used as a 
reference position to determine the amount of 
pronation and supination 

Position and mobility are estimated not 
measured 9/12 use categorical rather than 
numerical data 

Position is estimated not measured. 11/11 use 
this assessment in the frontal plane only 



the leg was always observed Foot type is 
classified as pronated/supinated/neutral This is a 
key biomechanical examination of the foot Heel 
bisection lines do not add value to the 
examination Podiatrists feel that they could 
accurately measure the frontal plane position of 
the calcaneus quantitatively if required 

Podiatrists state that the normal range of ankle 
dorsiflexion is 10° Assessment of the range of 
motion is based on the podiatrist's own 
experience as to what they perceive as normal 
and not through the use of a goniometer/other 
measuring device Podiatrists feel that they could 
accurately measure the range of ankle joint 
dorsiflexion quantitatively if required 

Podiatrists believe that this examination is a good 
indicator of dynamic foot function, but it is difficult 
to conduct 



Consensus from podiatrists was that for 
examination of first ray mobility and position 
categorical data (e.g. "rigid/flexible/normal") is 
more useful than numerical data 

No consensus on what should be used to define 
the forefoot (e.g. use middle three metatarsals or 
use all five metatarsals) 



Motion is estimated not measured 9/12 assess Consensus from podiatrists was that assessment 
the total range of motion of the first MTPJ 6/12 of the forefoot was dependent on the 



assess the range of first MTPJ dorsiflexion 



6/12 use the FPI as an assessment of foot 
type/posture 

All podiatrists assess the lower limb, 
leq and foot 



Limb length is estimated not measured 9/12 
assess anatomical limb length 7/12 assess 
functional limb length 



Examination of internal and external hip 
rotation Examination of hamstring flexibility 
(Straight leg raise test) "Heel raise" test 
to assess function of tibialis posterior 

11/11 assess the dynamic function of the foot, 
ankle and knee 10/1 1 assess the dynamic 
function of the hip and upper body 

Position of foot at heel strike Forefoot and 
midfoot position during loading phase.Foot 
position and motion during propulsion and 
re-supination Movement of the foot and leg 



presenting musculoskeletal complaint/injury and 
their focus was always on the function of the 
rearfoot 

Some podiatrists were unaware of the FPI 

Some podiatrists did use individual elements of FPI 

Podiatrists state that it is important to assess the 
pelvis, lower limb, leg and foot in a biomechanical 
assessment 

Consensus from podiatrists was that the 
examination of limb length is important and a 
limb length discrepancy is a common cause of 
abnormal biomechanical function of the foot, leg 
and lower limb Podiatrists feel that the process of 
obtaining a precise measurement (through tape 
measure) is not reliable and instead categorise the 
leg length discrepancy, for example <5 mm, 
<10 mm, >10 mm Measurement of limb length 
should also involve shoulder tilt, ASIS symmetry 
(supine and standing) 

Podiatrists state that these are not mandatory 
examinations and therefore are only used for 
specific clinical presentations 



Dynamic assessment is as important as static 
examination for diagnosis and development of a 
treatment plan 

Podiatrists state that they follow a relatively 
consistent protocol when conducting a clinica 
gait analysis assessment. The protocol involved 
identifying foot function at key events during the 
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Table 2 Results of Phase 1, 2 (questionnaires) and 3 (group discussion) on assessment of the foot and ankle (Continued) 



during swing phase Motion of the hip 
and knee Timing and magnitude of motion 
4 to 6/12 podiatrists had access to gait 
analysis equipment e.g. pressure plate, 
2D video analysis 



gait cycle and always aiming to analyse these from 

a visual perspective Consensus among podiatrists was 

that they would compare the dynamic function of a 

patient's foot and ankle to the description 

of "normal" they were taught at 

undergraduate level, the predominant basis for this 

was Root et al. [3,4] 

The consensus among podiatrists was that additional 
gait analysis equipment did not aid their assessment 
or treatment plan. All podiatrists felt they were 
confident in their visual analysis of the patient walking 
and what was feasible within the time constraints 



(ii) Podiatrists "estimate" rather than measure foot or 
limb position and motion. 

(iii) In addition to their static assessment, podiatrists 
conduct a dynamic gait assessment focusing on 
observation at key events of the gait cycle. 

The biomechanical assessment protocol identified 
through consensus comprised the following: 

(i) Examination of the foot in relaxed and neutral 
calcaneal stance position (RCSP and NCSP) 

(ii) Examination of forefoot to rearfoot relationship in 
the frontal plane 

(iii) Examination of the range of ankle joint dorsiflexion 

(iv) Examination of the position and mobility of the 
sagittal plane motion at the first ray 

(v) Examination of the range of sagittal plane motion at 
the first metatarsophalangeal joint (MTPJ) 

(vi) Foot Posture Index [31] 

(vii) Examination of limb length 

(viii) Visual gait analysis. Table 1 describes the 
protocols for the assessments chosen. 

Part 2 

The results indicate poor inter-assessor reliability for the 
four examinations. Table 3 displays the reliability results 
for RCSP and NCSP. For RCSP an ICC of 0.23 (right), 
0.14 (left) and 0.14 (right) and 0.11 (left) for NCSP sug- 
gest poor inter-assessor reliability. All mean 95 % CI 
were above 3.7° and the mean range of NCSP and RCSP 
values were greater than 8.8° (Table 4). The results of 
the random effects ANOVA indicate that the greatest 
error was random error (up to 4.9°), while the assessor 
error was up to 3.4°. 

Table 3 demonstrates ICC values for the examination 
of the range of ankle joint dorsiflexion. There was mod- 
erate agreement with 0.44 (right) and 0.42 (left) for knee 
extended and 0.61 (right) and 0.51 (left) for knee flexed. 
All mean 95 % CI were above 9.0°, and the mean range 
of ankle dorsiflexion values was greater than 20.5° 



(Table 5). The results of the random effects ANOVA in- 
dicate that there were comparable contributions from 
the three sources of error, with values ranging from 4.3° 
to 5.8°. 

The results for classification of first ray position and 
mobility are displayed in Table 3 and Table 6. There was 
greater consistency for the categorisation of mobility 
compared to first ray position. Fleiss Kappa values of 
-0.03 (right foot) and 0.01 (left foot) for categorisation 
of position and for the range of first ray motion (0.05 
(right) and -0.01 (left)). 

Table 3 and 7 demonstrates the results for examin- 
ation of limb length. There was less agreement on the 
size of the difference in limb length than the identifi- 
cation of the longer limb when evaluating the per- 
centage agreement values, however results were 
comparable according to Fleiss Kappa values (0.02 for 
both longer leg and the difference in leg length). Clin- 
icians consistently reported differences in limb length 
of 5 mm or less (Table 3 and 7). 

Table 3 Inter-assessor reliability results for all 
examinations 

ICC values for RCSP and NCSP 

RCSP (ICC 2'1) NCSP (ICC 2'1) 

Right foot 0.23 0.14 

Left foot 0.14 0.11 

ICC values for the range of ankle joint dorsiflexion 

Knee extended Knee flexed 
Right foot 0.44 0.61 

Left foot 0.42 0.51 

Fleiss Kappa values for categorisation of first ray position and mobility 

First ray position First ray mobility 

Right foot -0.03 0.05 

Left foot 0.01 -0.01 

Fleiss Kappa values for the categorisation of limb length examination 

Identification of Identification of 
longer leg longer leg length 

Limb length examination 0.02 0.02 



Jarvis et al. Journal of Foot and Ankle Research 201 2, 5:1 4 
http://www.jfootankleres.eom/content/5/1/14 



Page 7 of 10 



Table 4 Descriptive analysis of the variation between 
assessors in the examination of RCSP and NCSP 



Foot 


Examination 


Mean (°) 


sd n 


Range (°) 


95 % CI O 


Right 


RCSP 


0.2 


3.2 


11.2 


-2.0 to 2.6 


foot 


NCSP 


3.4 


3.6 


12.2 


0.9 to 5.8 


Left 


RCSP 


-0.4 


3.4 


11.2 


-2.7 to 1.9 


foot 


NCSP 


3.2 


2.8 


8.8 


1 .2 to 4.9 


Results of random effects ANOVA 






VEstimate of covariance parameter (°) 




Foot 


Examination 


VE random 


VE' assessor 


VE 2 subject 


Total 






error (°) 


error (°) 


error (°) 


0 


Right 


RCSP 


3.2 


0.6 


1.8 


3.8 


foot 


NCSP 


2.2 


2.9 


0.8 


3.8 


Left 


RCSP 


4.9 


3.4 


1.1 


9.5 


foot 


NCSP 


2.2 


1.8 


1.0 


3.1 



Discussion 

Biomechanical assessment protocol that podiatrists use in 
clinical practice 

The assessment protocol developed in Part 1 of this inves- 
tigation is largely a modified version of Root et al. [3]. The 
description provided by Root et al. [3,4] is still very much 
at the forefront of clinical assessment of foot 
biomechanics and the basis for clinical descriptors of foot 
function during gait. This demonstrates the continued in- 
fluence of Root et al. [3,4] and the strong effect under- 
graduate education has on subsequent practice. However, 
the inclusion of the Foot Posture Index [31] and use of 
visual gait assessment signify that podiatrists have adopted 
new assessment approaches that they deem to add value. 

Table 5 Descriptive analysis of the variation between 
assessors in the examination of the range of ankle joint 
dorsiflexion 



Foot 


Examination 


Mean (°) 


sd n 


Range (°) 


95 % CI (°) 


Right 
foot 


Knee 
Extended 


3.9 


7.0 


23.0 


-0.8 to 8.6 




Knee Flexed 


10.5 


7.3 


23.0 


5.6 to 15.5 


Left 
foot 


Knee 
Extended 


3.0 


6.6 


20.5 


0.1 to 9.1 




Knee Flexed 


7.5 


6.9 


22.2 


5.2 to 14.2 



Results of random effects ANOVA 



VEstimate of covariance parameter (°) 

Foot Examination VE random VE' assessor VE 2 subject Tota 

error (°) error (°) error (°) error (°) 



Right Knee 
foot Extended 



5.2 



Knee Flexed 4.5 

Left Knee 4.9 
foot Extended 

Knee Flexed 5.1 



4.6 



5./ 
4.3 



5.2 



10.7 



9.3 
8.0 



Table 6 Descriptive analysis of the variation between 
assessors in the categorisation of first ray position and 
mobility 

Percentage agreement values 

Foot Examination Plantarflexed (%) Neutral (%) Dorsiflexed (%) 
Right foot First ray position 55.0 31.5 13.5 

Left foot 62.0 30.0 8.0 



Flexible (%) 



Neutral (%) Rigid {%) 



Right foot First ray mobility 94.0 
Left foot 91.0 



1.5 
7.0 



4.5 
2.0 



This did not extend as far as the use of potentially valuable 
instrumented gait assessment methods (For example video 
analysis, pressure plate). 

Contrary to the specific instructions of Root et al. [3], 
podiatrists choose to estimate and classify joint position/ 
motion rather than ascertain a directly measured numerical 
value. For example, when assessing the ankle joint, podia- 
trists choose to estimate the range of dorsiflexion rather 
than use a goniometer. Podiatrists felt that their experience 
was sufficient to accurately classify the range of motion as 
normal, excessive or restricted. All podiatrists stated that 
they were confident this approach was valid and cited time 
constraints as the primary barrier to use of objective mea- 
sures. However, continuing to use assessments that have 
been shown to have low reliability is likely to be considered 
unsound practice. If reliability could be improved by an ob- 
jective rather than subjective assessment, even if it takes 
longer to complete, then this could form a strong case to 
extend the time available for the assessment of patients. 

These differentiations from the original description and 
instructions of Root et al. [3,4] justify the consensus exer- 
cise in Part 1 and ensure that our investigation of inter- 
assessor reliability is relevant to current clinical practice. 

Inter-assessor reliability 

There was poor inter-assessor reliability recorded for all 
of the static biomechanical examinations of the foot, leg 
and lower limb which questions their value in clinical 
practice. RCSP and NCSP produced poor inter-assessor 



Table 7 Descriptive analysis of the variation between 
assessors for the categorisation of limb length 
examination 



Percentage agreement values 


Examination 


Right (%) 


Left (%) 


None (%) 




Identification of 
longer leg 


64.0 


12.0 


24.0 






up to 5 mm 
(%) 


5-10 mm 
(%) 


greater than 10 mm 
(%) 


None 
(%) 


Identification of 


23.0 


39.0 


14.0 


24.0 



longer leg length 
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reliability results (all ICC values were less than 0.23), 
and this concurs with the available literature. Picciano 
et al. [32] recorded ICC values for NCSP of 0.15 and 
95 % confidence intervals of 0.87° to 8.65°. The results of 
the random effects ANOVA suggest that for RCSP and 
NCSP random error is the key issue. Differences be- 
tween assessors of this scale would create different treat- 
ment plans and orthotic designs. Both Menz [8] and 
Elveru et al. [16] highlight that an overwhelming priority 
is placed upon the outcomes of these measurements in 
clinical assessment and orthotic prescription. However, 
the poor reliability and large variation in the results 
recorded here and elsewhere [8,9,12,16] should be clinic- 
ally unacceptable and we therefore question their con- 
tinued use in clinical practice [8,15,16]. 

Although podiatrists reported some difficulty in using 
the goniometer [15], moderate reliability was observed 
for the examination of the range of ankle joint 
dorsiflexion. Elveru et al. [16] and Jonson and Gross [1] 
report similar ICC values of 0.50 and 0.65. In Part 1 of 
this study all podiatrists stated that they believed the 
examination of ankle joint dorsiflexion provided a good 
indication of dynamic foot function. However, the low 
reliability and large range of values recorded across 
assessors questions the clinical value of these examina- 
tions. Considering that 10° of dorsiflexion was stated as 
normal (results from Part 1, based on Root et al. [3], 
Table 2), clinical measures at either boundary of the 
95 % CI (maximum 95 % CI were 5.6° to 15.5°) and the 
total error of up to 10.7°, could lead to false identifica- 
tion of the actual range of ankle dorsiflexion. This 
would directly affect the treatment rationale if the out- 
come suggested limited or adequate range of ankle 
motion. Moseley and Adams [33] suggest that such vari- 
ation would make measurement of changes in range of 
motion due to interventions (e.g. stretching) unreliable. 
The results from the random effects ANOVA suggest 
that all three sources of error contribute to variation be- 
tween assessors. Since random error was quite large 
(5.2°, left foot, knee flexed), reducing errors from parti- 
cipants and assessors (e.g. through training, use of 
measurement tools) might still not achieve an accept- 
able level of reliability. 

Classification of first ray mobility demonstrated greater 
reliability than categorisation of first ray position. The 
Fleiss Kappa values of less than 0.05 for categorisation of 
first ray position and range of motion indicate only poor 
to slight agreement [28,29]. For four of the 12 feet 
assessed there was greater than 90 % agreement for classi- 
fication of first ray range of motion as flexible. However, 
percentage agreement can lack sensitivity as to the true 
level of agreement between assessors as it can over or 
under estimate the actual level of agreement and does not 
account for the possibility that the agreement observed 



occurred by chance [27]. High levels of agreement for as- 
sessment of flexibility might be expected as 'rigidity' sug- 
gests no motion at all and this is more easily identified 
than different grades of "some" motion [24] . However, tak- 
ing into account the Fleiss Kappa and percentage agree- 
ment statistical values only poor to moderate reliability 
was observed. Classification of first ray position demon- 
strated poor agreement between assessors. There are sig- 
nificant identifiable differences between a plantarflexed 
and dorsiflexed first ray [24], something that experienced 
podiatrists would expect themselves to be able to identify. 
As with measures of rearfoot alignment, first ray position 
can influence orthotic prescription [24] . 

Identification of the longer limb provided marginally 
better agreement than classifying the actual amount of 
leg length difference, but still only suggests slight agree- 
ment [28,29] with Fleiss Kappa values of 0.02 (longer 
leg) and 0.02 (difference in leg length). This level of reli- 
ability is similar to Woerman and Binder-MacLeod [21]. 
To be able to ascertain that there is a difference in limb 
length of less than 5 mm requires high precision and it 
is doubtful that through visual inspection and palpation 
a clinician could reliably work to such accuracy. If a clin- 
ician can identify a discrepancy this small then they will 
almost always identify a limb length difference because 
the skeleton is rarely truly symmetrical. 

Clinical implications 

One purpose of clinical assessment is to decipher nor- 
mal from pathological [1,2,23,24] but the results from 
this investigation suggest that it would not be possible to 
accurately classify either. The protocol described by Root 
et al. [3] states precise measurements are required when 
undertaking a static biomechanical assessment of the 
foot. Results from this and prior research [11,12,16,32] 
suggest that such accuracy is not achieved in clinical 
practice. For example, Root et al. [3,4] states that RCSP 
and NCSP measurements will precisely dictate the in- 
clination of a rearfoot wedge used in a foot orthoses. 
However the variability in the assessment of rearfoot 
position reported here would lead to very different 
orthotic prescriptions. This directly undermines the 
biomechanical rationale for intricate adjustments in the 
design of foot orthoses and the capture of static foot 
shape as a basis for foot orthosis design. This has pro- 
found implications for many areas of clinical practice 
and suggests a reappraisal of the theoretical and prac- 
tical basis for orthotic practice is warranted. The low re- 
liability of the assessments evaluated here questions 
their ability to accurately infer the behaviour of the foot 
during stance, which is the purpose of the static assess- 
ments in the model proposed by Root et al. [4]. Indeed, 
research investigating the validity of Root et al. [3,4] is 
currently being undertaken by the authors. The results 
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here also add weight to the case for a move toward ob- 
jective assessment of dynamic foot behaviour in clinical 
practice, regardless of the practical challenges this raises. 

Limitations 

There are several limitations to the work reported here. 
Four of the eight examinations used by assessors (from 
Part 1) were not included in Part 2 of this study. They 
were excluded because the podiatrists we worked with 
identified them as 'optional'. Other clinicians might 
disagree with the ranking of the eight assessments, espe- 
cially if their practice is different to that of the podia- 
trists involved in this current study. Using all eight 
examinations would have been logistically difficult with 
the number of assessors and participants in this study 
and time available for the assessments. The number of 
assessors used was relatively small and might not repre- 
sent the true variation across the entire professional 
communities using the assessments evaluated in this 
work. All were podiatrists and whilst their professional 
networks are strongly multi-professional, practices could 
differ in other disciplines and countries. The literature 
indicates that the measures used by the assessors and 
those evaluated in the reliability study, are also used in 
the physical therapy profession [1,16,20]. The develop- 
ment of the foot assessment protocol occurred through 
just one iteration of the Delphi method, whereas two or 
more iterations are often employed [24]. Experience dur- 
ing the exercise suggested that consensus was already in 
place or very close from the outset. The number of feet 
assessed was quite small and all participants were free 
from pathology. The participants were young with an 
average BMI and may not represent feet that present in 
many clinical cases. Arguably, assessing these feet is eas- 
ier than those of people in pain, feet with deformity or 
in cases of greater BMI, and thus our results might re- 
flect a "best case" scenario in terms of reliability. This 
study recorded low ICC values, in particular for NCSP 
and RCSP. The large number of assessors and small 
number of participants would have increased the vari- 
ability and therefore could have decreased the inter- 
assessor reliability. Finally, good reliability does not infer 
practical usefulness of the assessment. Good reliability 
may simply reflect low sensitivity and specificity in the 
measure, or highly repeatable errors by assessors. Thus, 
good reliability does not infer validity. However, mea- 
sures cannot be valid unless reliable, and outcomes of 
this work indicate many of the assessments used in foot 
health practice are unreliable and thus invalid. 

Conclusions 

Static biomechanical assessment of the foot, leg and 
lower limb is considered important in clinical practice. 
The key examinations used to make inferences about 



dynamic foot function, to construct a treatment plan 
and to determine orthotic prescription are unreliable. 
Using these examinations to differentiate normal from 
pathological foot function would not appear to be valid 
clinical practice. 

Additional file 



Additional file 1: The questionnaire used for the modified Delphi 
technique investigation in Phase 1, Part 1. 
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